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EXAMINER'S ANSWER 



This is in response to the appeal brief filed August 9, 2007 appealing from the 
Office action mailed September 19, 2006 
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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial 
proceedings which will directly affect or be directly affected by or have a bearing on the 
Board's decision in the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is 
correct. 
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(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 

6,292,574 Schildkraut et al. 09-2001 

6,181,806 Kadoetal. 01-2001 

5,410,618 Fowler 04-1995 

6,680,745 Center, Jr. et al. 01-2004 

6,009,209 Acker et al. 1 2-1 999 
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(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 



Claim Rejections - 35 USC § 103 



The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

A patent may not be obtained though the invention is not identically disclosed or described as set forth 
in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art 
are such that the subject matter as a whole would have been obvious at the time the invention was made to a 
person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. . 



Claims 1, 2, 8, 10, 15, 21 and 27-33 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Schildkraut et al. (US 6,292,574), in view of Kado et al. (US 
6,181,806) and Fowler (US 5,410,618). 



Regarding claim 1, and similarly claims 8, 15 and 21 (all as amended), 

Schildkraut discloses: 

• automatically detecting one or more human faces in an image using face detection algorithms 
and automatically locating the one or more human faces in the image 
[Fig. 2, numeral S10; Fig. 6; Col. 4, line 13-Col. 5 t Iine4] 



Schildkraut does not expressly disclose 

• automatically enhancing an appearance of the entire image by using a mapping technique to 
produce the image with target levels for a mean value or a variation value of the pixels in the 
one or more human faces, wherein the entire image is automatically enhanced such that the 
pixels in the one or more human faces have the target levels for the mean value or the 
variation value of the pixels 
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However, Kado discloses automatically measuring brightness (i.e., lightness 
levels) of human faces [Fig. 14, ref. 16 and col. 7, lines 33-36] and automatically 
enhances the image's brightness based on the measure brightness [col. 7, lines 44-46]. 
In addition, Fowler discloses linear mean invariant transforms (i.e., a mappings) that 
enhance lofargram images in such a way that the each and every strip of the image is 
enhanced to have a target level of mean value [Fig. 1, refs. 3-7; Col. 1, lines 30-40; Col. 
3, lines 15-30; Col. 6, line 48-Col. 7, line 5. Note that here Fowler discloses enhancing 
all regions of an image such that each region has a target level for the mean value; in 
the case of an image containing faces, all facial regions and the non-facial region as a 
whole are all regions of the image (possibly as a result of some segmentation process). 
Therefore Fowler teaches enhancing the entire image.] [For demonstration purpose 
(but not relied upon for rejection) an example of a lofargram image can be found in the 
first slide of the presentation made by Vincent Premus in the 7 th Annual ASAP '99 
Workshop.] 

Schildkraut, Kado and Fowler are combinable because they all have aspects that 
are from the same field of endeavor of image enhancement (note that red eye 
correction is a form of image enhancement). 

At the time of the invention, it would have been obvious to one of ordinary skill in 
the art to modify Schildkraut with the teachings of Kado and Fowler to enhance detected 
human faces by using a mapping technique. The motivation would have been to 
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improve visual quality, among other things, as Fowler indicated in [Col. 1 , lines 1 1-20 
and Col. 2, lines 28-35]. In addition, as the purpose of red eye correction is to provide 
an facial image with a better visual quality, it would have been obvious to go one step 
further by enhancing the overall visual quality of the faces (and not just the eyes) in the 
image since doing so would provide even better results to the clients (e.g., by a photo- 
processing lab to its customers). 

Therefore, it would have been obvious to combine Kado and Fowler with 
Schildkraut to obtain the invention as specified in claim 1. 

Regarding claim 2, and similarly claim 10, Kado further discloses 

• wherein the module for enhancing the appearances of the image includes a module for 
automatically enhancing lightness levels of the images to enhance the appearance of the one 
or more human faces 

[Fig. 14, ref. 16 and col. 7, lines 44-46] 

Regarding claim 27, and similarly claims 28 and 29, Schildkraut further discloses 

• automatically locating eyes in the human faces 
[Fig. 2, numerals S16-S30; Figs. 9, 1 1] 

Regarding claims 30 and 31, Fowler further discloses mapping using the recited 
equations. [Col. 6, line 48-Col. 7, line 5. Note that Eq. 13 is a more general mapping 
function that includes both scaling and offsetting. When only offsetting is performed, 
i.e., A = 1, Eq. 13 becomes P + B (note that in line 1 of col. 7 ¥ should have been V). 
On the other hand, when only scaling is performed, i.e., B = 0, Eq. 13 becomes 
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AP + (1-A)u. Note further that B is the difference of the new and the original means and 
A is the square root of the ration of the new and the original variants, per Eqs. 1 1 and 
12 in col. 6] 

Regarding claims 32 and 33, note that as admitted in page 5, lines 13-18 of the 
specification, determining the target levels through a determination of human visual 
preferences is well known in the art and would obviously have been considered by one 
of ordinarily skill in the art since facial images, especially photographs, are typically 
viewed by humans. 

*** 

Claims 3, 4, 11, 12 and 18 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Schildkraut etal. (US 6,292,574), Kado etal. (US 6,181,806) and 
Fowler (US 5,410,618) as applied to claims 1, 2, 8, 10, 15, 21 and 27-33 above, further 
in view of Center, Jr. et al. (US 6,680,745). 

Regarding claims 3 and 4, and similarly claims 11,12 and 18, the combined 

invention of Schildkraut, Kado and Fowler discloses everything except the following, 

which Center teaches: 

• wherein the module for enhancing the appearances of the image includes a module for 
automatically enhancing contrast levels (claim 3) or color levels (claim 4) of the images to 
enhance the appearance of the one or more human faces 

[Fig. 1; col. 2, lines 55-60. Note that while Center discloses adjusting the camera's settings, 
one of ordinary skill in the art would have recognized that such changes can be achieved by 
image processing means; see, for example, the analysis of claim 2 above in which Kado is 
relied upon for enhancing brightness] 
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The combined invention of Schildkraut, Kado and Fowler is combinable with 
Center because they have aspects that are from the same field of endeavor of face 
detection. 

Therefore, at the time of the invention it would have been obvious to one of 
ordinary skill in the art to modify the combined invention of Schildkraut, Kado and 
Fowler with the teaching of Center by enhancing an appearance of the image by 
changing the contrast and/or color (in addition to changing brightness as Kado 
disclosed). The motivation would have been to obtain better images, as Center 
indicated in column 2, lines 55-60. 

Therefore, it would have been obvious to combine Center with Schildkraut, Kado 
and Fowler to obtain the inventions as specified in claims 3 and 4, respectively. 

*** 

Claims 6, 14, 20 and 25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Schildkraut et al. (US 6,292,574), Kado et al. (US 6,1 81 ,806) and 
Fowler (US 5,410,618) as applied to claims 1, 2, 8, 10, 15, 21 and 27-33 above, further 
in view of Acker et al. (US 6,009,209). 
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Regarding claim 6, and similarly claims 14, 20 and 25, the combined invention of 
Schildkraut, Kado and Fowler discloses all limitations except the following, which Acker 
et al. teaches: 

• reducing or removing the red eye artifact from the human faces 
[Fig. 5, numeral 109; Fig. 9, numeral 504; Fig. 11; Fig. 13] 

The combined invention of Schildkraut, Kado and Fowler is combinable with 
Acker because they have aspects that are from the same field of endeavor of red eye 
detection. 

Therefore, at the time of the invention it would have been obvious to one of 
ordinary skill in the art to modify the combined invention of Schildkraut, Kado and 
Fowler with the teaching of Acker et al. by detecting and removing red-eye effects from 
the image. The motivation would have been to remove the unpleasant appearance of 
red-eye defects in an image of a person's face caused by, e.g., a flash when the image 
was taken in order to produce a more natural-looking face. 

Therefore, it would have been obvious to combine Acker et al. with Schildkraut, 
Kado and Fowler to obtain the invention as specified in claim 6. 
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(10) Response to Argument 

(Note: In the following the cited page numbers of the appeal brief are the page 
number shown on upper-right corner of each page of the fax received on 
08/09/2007 since the original page numbers are not obscured.) 

Issue A: Rejection of claims 1, 2, 8, 10, 15, 21 and 27-33 based on Schildkraut et 
al., in view of Kato et al. and Fowler. 

Appellant's arguments regarding Issue A are not persuasive. Specifically: 

A. 1 Appellant argues that because the mean and variance of the frequency in 
another vertical frequency strip have no effect on the pixels of the particular 
vertical frequency strip, the pixels of different strips may be modified to have 
different levels of enhancement and accordingly the entire lofargram image is not 
enhanced to make one of the vertical frequency strips have target levels for the 
mean value or the variation value of the pixels contained in any one of the 
vertical frequency strips (P. 11, 3 rd paragraph-P. 12, 2 nd paragraph) 

Examiner's response: 

Claim 1 sets forth that an image is produced "with target levels for a mean or a 
variation value of the pixels in the one or more human faces... such that the 
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pixels in the one or more human faces have the target levels for the mean or the 
variation value of the pixels." In other words, each human face has a target level 
of mean or variation after the enhancement, which is exactly what Fowler 
teaches [Fig. 1, refs. 3-7; Col. 1, lines 30-40; Col. 3, lines 15-30 (the new, 
enhanced mean and variance are the target levels for each of the strips in the 
enhanced image); Col. 6, line 48-Col. 7, line 33; with each strip being a region of 
the image]. Note that claim 1 as recited does not require that the mean or 
variance of the pixels in one or more human faces (i.e., one or more regions of 
the image in question) to have any effect on the pixels of another region. In 
addition, Fowler clearly teaches enhancing the entire image since each and 
every strip is enhanced. Note further that even if all the faces are enhanced to 
have a common target level of mean or variation, Fowler's teaching still applies 
since the collection of all faces in an image is nothing but a separate region of 
the image and the pixels within that region can be mapped (enhanced) in such a 
way that they collectively have a target mean or variation. Therefore the 
argument is not persuasive. 

A.2 Appellant argues that the references cited by the examiner fail to at least 
disclose that the entire image is enhanced (P. 13, 2 nd paragraph) because 
Schildkraut et al. and Kado et al. fail to disclose it (P. 13, 3 rd paragraph; P. 14, 2 nd 
paragraph) and Fowler does not cure this deficiency (P. 15, 2 nd paragraph) 
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Examiner's response: 

First, enhancing an entire image does not necessarily mean that each and every 
pixel value has to be altered (not a claim requirement, nor expressly defined in 
such a way in the specification). The pixels in a portion of an image can be 
altered in such a way that the entire image is considered enhanced since it 
results in a more, say, pleasing overall visual effect. An example is the red-eye 
correction (in which the unpleasant red eyes due to the flash of the camera is 
corrected). Nonetheless, Fowler clearly teaches enhancing the entire image 
since each and every one of the strips in the lofargram (an image) is enhanced 
[Fig. 1, refs. 3-7; Col. 3, lines 15-30]. Whether applying Fowler's teaching will 
result in various sections of the image containing the human faces being 
separately enhanced (as appellant alleges in P. 15, 2 nd paragraph, last three 
lines) is irrelevant since claim 1 does not require all faces to be enhanced as a 
whole and even if it does, the collection of face sections can be considered as a 
single region (namely, a "face region") and the teaching of Fowler still applies. 

A. 3 Appellant argues that none of the cited references discloses that the target 
levels for a mean or a variation are desirable lightness and contrast levels that 
are determined through a determination of human visual preferences (P. 15, 2 nd 
paragraph through P. 18, 2 nd paragraph) 



Examiner's response: 
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Examiner's response: 

However, Kato does disclose that the pixel values that are to be enhanced to 
achieve target levels for a mean are lightness values [Fig. 14, ref. 16 and Col. 7, 
lines 44-46]. Note also that when lightness is adjusted, contrast may change 
accordingly. In any event, per the analysis in the rejection of claims 30 and 31 
(see section 9 above), Fowler also discloses adjusting the pixel values to achieve 
a desired variance value, which results in a different contrast. Additionally, using 
target levels for a mean or a variation that are desirable lightness and contrast 
levels determined through a determination of human visual preferences is known 
in the art, as appellant admits in page 5, lines 1 3-1 6, and one of ordinary skill in 
the art would have a good reason to do so since it typically would have been a 
human being who would be viewing the digital images as shown in, e.g., figure 1 
of Schildkraut et al. (Note: Appellant made a similar argument in the amendment 
filed 03/23/2006 and this response is essentially from the office action mailed 
05/03/2006, in response to this argument.) 

A. 4 Appellant argues that the proposed combination is improper because (1 ) 
Fowler is directed to sonar images and both Schildkraut et al. and Kado et al. are 
directed to facial images and as such it appears likely that the proposed 
combination of documents was based on improper hindsight reasoning and (2) 
that the proposed combination would still fail to disclose each and every element 
claimed in claims 1,8, 15 and 21 (P. 18, sect, d); as well as (3) that one would not 
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have been motivated to combine the references (P. 19, 1 st paragraph) and (4) 
that there is no clear indication as to what the proposed combination would yield 
(P. 19, 2 nd paragraph) 

Examiner's response: 

Regarding (1) and (3), in response to applicant's argument that the examiner's 
conclusion of obviousness is based upon improper hindsight reasoning, it must 
be recognized that any judgment on obviousness is in a sense necessarily a 
reconstruction based upon hindsight reasoning. But so long as it takes into 
account only knowledge which was within the level of ordinary skill at the time the 
claimed invention was made, and does not include knowledge gleaned only from 
the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 
443 F.2d 1392, 170 USPQ 209 (CCPA 1971). In this case, all three references 
have aspects that deal with the enhancement of digital images and the admitted 
prior art [P. 5, lines 13-16 of the instance application] discloses that there are 
desirable levels for a mean value and/or a variation value that people prefer, and 
Fowler teaches how this can be accomplished [Fig. 1 , refs. 3-7; Col. 1 , lines 30- 
40; Col. 3, lines 15-30 (the new, enhanced mean and variance are the target 
levels for each of the strips in the enhanced image); Col. 6, line 48-Col. 7,. line 
33; with each strip being a region of the image] and the result is improved visual 
quality [Col. 2, lines 28-35]. Therefore the argument is not persuasive. 
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Regarding (2), it is not persuasive because all limitations have been disclosed or 
taught by the combined references; see the analysis and rejection of claims 1, 2, 
8, 10, 15, 21 and 27-33 (reproduced in section 9 above) as well as A.1-A.3 of this 
section. Therefore the argument is not persuasive. 

Regarding (4), note that, for example, the combined invention of Schildkraut et al. 
Kado et al. and Fowler will predictably result in the invention as specified in claim 
1 since Schildkraut et al. discloses a method that automatically detects and 
locates faces in an image and performs a kind of facial enhancement (namely 
red-eye correction), Kado et al. teaches automatic enhancement of the 
appearance (brightness in this case) of human faces, and Fowler teaches how 
the enhancement is carried out (namely by using a mapping technique to 
produce an image such that the different sections of the image have target levels 
of mean or variation). Again, see the analysis and rejection of claims 1,2,8,10, 
15, 21 and 27-33 (reproduced in section 9 above) as well as A.1-A.3 of this 
section. Therefore the argument is not persuasive. 

Issue B: Rejection of claims 3, 4, 11, 12 and 18 based on Schildkraut et al., Kato 
et al. and Fowler and further in view of Center, Jr. et al. 
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Appellant's argument regarding Issue B is not persuasive. Specifically: 

S. 1 Appellant argues that the Center, Jr. et al. does not make up for the 
deficiencies in Schildkraut et al., Kato et al. and Fowler (P. 20, sect. 2) 

Examiner's response: 

Center, Jr. et al. is not relied upon to make up for the deficiencies in Schildkraut 
et al., Kato et al. and Fowler. 

Issue C: Rejection of claims 6, 14, 20 and 25 based on Schildkraut et al., Kato et 
al. and Fowler, and further in view of Acker et al. 

Appellant's argument regarding Issue C is not persuasive. Specifically: 

C. 1 Appellant argues that the Acker et al. does not make up for the 
deficiencies in Schildkraut et al., Kato et al. and Fowler (PP. 20-21, sect. 3) 

Examiner's response: 

, Acker et al. is not relied upon to make up for the deficiencies in Schildkraut et 
al., Kato et al. and Fowler. 
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(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the 
Related Appeals and Interferences section of this examiner's answer. 



For the above reasons, it is believed that the rejections should be sustained. 
Respectfully submitted, 
Yubin Hung 




Conferees: 
Matthew C. Bella 



Jingge Wu 



MATTHEW C. BELLA 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 



